How to avoid exploratory research

Introduction: Studies in marketing research often start with data rather than with a theory. This exploratory or inductive approach is at odds with the more preferred scientific method where the theory precedes the data in any single research study. (See, for example, the discussion by Francis, 1957) Because exploratory research is common, however, one might argue that it is of some value. A number of researchers have claimed that the exploratory approach leads to new and useful theories. But there is also the danger that the research will produce false leads or useless theories. An attempt is made in this paper to illustrate the dangers inherent in the exploratory approach. The question of whether the potential benefits are large enough to outweigh the dangers is left to the reader. Disciplines Public Relations and Advertising Comments Postprint version. Published in Journal of Advertising Research, Volume 10, Issue 4, 1970, pages 27-30. The author has asserted his/her right to include this material in ScholarlyCommons@Penn. This journal article is available at ScholarlyCommons: http://repository.upenn.edu/marketing_papers/4 Published in Journal of Advertising Research, 10 (4), 1970, 27-30. How to Avoid Exploratory Research J. Scott Armstrong Studies in marketing research often start with data rather than with a theory. This exploratory or inductive approach is at odds with the more preferred scientific method where the theory precedes the data in any single research study. (See, for example, the discussion by Francis, 1957) Because exploratory research is common, however, one might argue that it is of some value. A number of researchers have claimed that the exploratory approach leads to new and useful theories. But there is also the danger that the research will produce false leads or useless theories. An attempt is made in this paper to illustrate the dangers inherent in the exploratory approach. The question of whether the potential benefits are large enough to outweigh the dangers is left to the reader. One of the more popular techniques which has been used for exploratory research is regression analysis, in particular, a stepwise version. The following example illustrates the exploratory use of stepwise regression. Tom Swift, a researcher for the International Caribou Chip Co., was asked to study the international market for caribou chips. Management was interested in some explanations as to what caused variation in sales levels among countries – e.g., why were sales higher in France than in Germany? The company had reliable data on industry sales for caribou chips for 31 countries and Swift conducted a search for possible explanatory variables. Fortunately, the quality and accessibility of international data have been improving rapidly over recent years (e.g., see Russett, 1964). As a result, it was possible to obtain data on 30 variables which might have some relationship to sales of caribou chips. The task then was to determine which of these 30 variables were, in fact, most closely related to the sales level in each country. The sales rates in each country were regressed against the explanatory variables by means of a stepwise regression program, using the UCLA Biomedical O2R program (Dixon, 1967). This is a "step-up program" – i.e., the variable having the highest gross correlation with the dependent variable is first entered into the regression. Next, the variable with the highest partial correlation (controlling for the variable already .in the regression is entered), etc. Only variables whose coefficient had a t-statistic of greater than 2.0 were retained in the model. (Swift interpreted this as implying that there was-less than a five per cent chance that the true coefficient was zero.) Three of the 31 countries proved to be "outliers" on the initial regression analysis. The deviation between actual and predicted sales for each of these three countries was so large that there was less than a 10 per cent probability of such an event occurring by chance. A careful examination of these three outliers led to the conclusion that they were not comparable with the 28 other countries, and they were dropped from the analysis. The regression was then reanalyzed. The effect of dropping the three countries was to increase the coefficient of determination, R, by about 0.15. The second, and final, regression run is presented in Figure 1. The last three variables did not contribute substantially to the fit of the model (in total they increased R by about 0.10), but Tom was following his prespecified rule of including all variables where the t-statistic was greater than 2.0.